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m&mit^mz, ffls&xtj¥8r?\t>-£ftit$m<t/< io 

7^%ttiC ffi3BlM«K0ra^fk7*-*®«^ • 

[!SsRiR2] ffllEft#{b'*7*-*tt, I««©flHt 
{k7U-A0«^iefre>^ie^©Mtf§*.®'^*- 

Ho 

mmm%<D^\tT~^\c^mtftxx-ht 

<D/^y7 7i:, 

C0«»©>W7 7 0*n€ftfc^£ftfcft#{kT- 

**y*5rv MbLT£»b"f 3£»b¥K^ 

c (Dzmit^m-ezmit-g ftxm ftftzmtT- $z 30 

fcJMSU 

w^mm %m , fi 12^ >y 7 7 tc -iss* $ n & ft # 

qs 4 *> t# S RHf { kr - * o fflf B> W 7 r 0 **S 

-* t^ptDnmtT-^^'tft^ft-mmmt^m 

£>/^y7 7fc* 40 

c o«»o/vy 7 7 ntft=e fticmmz ftitftmtr- 
c (D&mit^mxzmitzftxm 6 ntczmt?- 2 * 

mzzmitrnm, mmmt7-z*Mmt% c tic 
< b^aa^ s * ^ s s c i: i: -r 



WfWl s F9-2 4 7 6 7 0 

2 

[00 0 1] 
[00 0 2] 

[ce*©afB] »iffl«oE*t^fk35rsoBiB«ttfcu 

fcl/ftMPEG-l ( I S 0/ I E C 1 1 1 7 2) 
&t&ft0r5>*;Uttj& VOD (Video On D 
emand) DVD (Digi ta I V 

i d e o Disk) fc^'NOJEffltfWfSSnT^SM 
PEG-2 (I SO/I EC 1 38 1 8) ftif^306 

[0 0 0 3] M P E GffiF#{k#5£T(i» KM, & 

^ku zftzmmtT-zzzn^ft^'rv ntL 

X, b£S{k*fT*3£a£tfHUHSnT^So $ 

[0 0 0 4] T3- KBSMRtfS^I$8J0£r*-1'i»** 

sis> ^s-n^moflrefk/w- 
7 u- a u t ft^ktf s n a c n 5 © 

*w^kT*t4, a^«stw#<k«i:^a*o» sfe*© 
M^idWtcSfk'rsit^^i&So ^-oj: 9 - 

xiiiEft%tt^tT-z<D'\vwmmfrt>. muT. 

[0 0 0 5] iMfiffliJ-eW, SflUHJ-POA^I^y 

WJWR. RZf?-U±Xt>y7<D&m*fi&&ftVftli 

[0006] /N-K->x7T«$ns^M{kS 
H-ptt, £ftt><D$^L7>$'v7tt. %m{mmm<D$ 

D -y * OH W/^ISl^tC J; -3 TA £ < ^£ 5 1 o i; 4 
[0 0 0 7] 

[«W^»Ssb«fc7i:-rs»IH] MPEG^b^S^ 
X*«n*^kf-^€> / N-y^flHHO*^6» 



(3) 
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5 fc ate g^ii t nmtm t <D?n<DWL±\m 

[0 0 0 8] 111, ^#©#tS$SMfr6©tit$R£ffl 
££B L 1 XitLtzZMit t£ -y h X h'J - A£f#3 S 1? 

©^rojaste&^T, ^— o^'f'^^-X'xoiRi^^ 

{5^LfctSfB^SfkSHTti, R*SJ4:flre<ke«y h»*J 10 
«fctfft#fkl/- h tti. 5t#te 1 *t 1 cDW{£Wt£f 3 
CfctfBTIilfcfc*. fct, ^SfbSBtefcttS^-y 

r- * * h U - L fr<b m 6 n« (SiB W4 9 4 L^-X te 

govt, »awt!:w-»-r5ci:A^piffii:aso 

[0 0 0 9] C©«fc5ftW«£gfkgHte*5^TM\ £ 

ffi{bffigteV7h7x7^S0#T'i^Wf££ft , 3, % 

T-fioiRfflttcos. % ->xf .ijisia* Rifii ^ ft £<> 

[0 0 10] ^fflH*ff^^V7h^x7 20 

it, '>ft< fc&^flHHkl/-h«fc!)fe+$HI^<:i:tfg 
jR^nSo tot, £Sfk«jatt><-xhtofcft&fc 

ft, V7F7x7±©ffiS!ftft#'l'2^-X£:il£'i'A 

©Hie 14, -y £ 5 C teft So 

[0 0 1 1] £<OV>y*tt. ilflt, il/Vy77T'M 

^{k*fT ft >*-y ^ r a®fi%s*/j^ < -r s 

CfcWBtiS. Sfc, SfBiJtefc^Tt, S?«y*!R 
iK©7cto<DA7VW7 7)I3i££, &®W±te**< £3 30 
c£te{if3:U<ft</\, ^-cDfcto, SffUJtefc^TATJ^ 
*y7 7iI5£*S'NkU ffirefkr-** h 'J-A<DA7J 
RSaifr 6 , iii»<D«[#ftf^*K»&-r S C i: ^riss-r 5 
£, £Sfk»0^*^A^-*fc^H#8Jfc01*n©fc 
fete, £»tt? <y F U-A0£&M&B3aj©ffifi!IL 

[0 0 1 23 fCT?, *fgB^«, w±©raiBSKaE*T 
ft^nfctoTfto, sunt, ^oteora^fkf* 
-*©«^{ki$*iJ, g^^MtePi-s^k/^y- 40 

S-O'vy #tt«fctt^fc£fifkS5 / \A*J*S C 
fcfcJ: t), 'N-y ^ttfSOlSWfOfcfefc^US^S^kilji 

[0 0 13] Sfc, £B{tSP<DA7j;W7 7 

0««*^H^J*«ISU, £fifbtf >y FT, h V-hm 

ff*, ±lE©7V^-7n-*l» «Jift/\ 
- F 7 x 7 & £ L ft i/ M* $H£» kaKfcflHft-f 3 c 

^^gWt-TSo 50 
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[0 0 14] 

'>ft< fctldfflifiROWF^k-r-^fcSjSOW^kr 

-^*/^-y hitbrzmittz&mit^mt, cz>& 

ft#{k><5*-**ArtT*7J3^8i:££ffiU MIS 

£S<k^8«, tfflBArt¥8?Afl«ftfcflF#fk/<5* 
cD^imamiB^wtBbT, ^ft^iwfieiifjiHiwffi^b 

tote^N >y ^1H$H<Dfi?t/Tte J: 9 £ D 3 §^S{ka® =&K y >T? 
[0 0 15] $fc, #3SIJl©tt«£Sfk8Hti» A7j£ 

n^>ft < a feiftiBftoWF^fkr-* fcSjsc^kx 
^cD/^>y7ro^n ; e*nte^tt$nfc^b7--^^/^ 

mtzmmb. tuK^fifb^sti, Buie^-y7 7te-BS 

Sasns^fkr-^O^-6, W8B/Vy7 7©i5Wfi 

o«»*^iaa* t ft o f§ s ^<br- ^ cottar * v 7 
7© I 5Wffl^ate, zmmmitTmtzm^yyz 

&fet%z.t\z.&K>* V7 h7x7teJ:SII^P^ma^ 
te fr A , ^^iJoSfi^ ^y 7 7 te ct £ 7 > ^- 7 
p-^K<"ti:tte, |g$dft/N-K7x7^^gt Lft 

[0 0 16] $/c, #»W©1*fR£Sfk&IIl4, ATJ^ 

nfOft < t tiftiittMo^k-r-* fcSjsc^kr 
-z* : tft?in--mwffi?%tm(DJ^y7 7 1, com 

SO/ Vy 7 7 ® ^tt^nfcSfcS ttfcffirefkr - * 
^r«y MbLT^BfbT 5£»b^K£, Z<D$mt¥& 

»ffi%atc^fifkftyi*fflj&'rs*^5yy*i*^-rs 

Cfc »c J: ») , V7b7x7(a5 ^^raBffl^sa t tT 

^UoSfg>' , ;'y7 7tefc(t57>^-7a-«rK 
<-t£&te, ISiiftM-F7x7^Si:Lft^o t^o 

r, ttf«<D^k, (S^k©n«fBasi!i:, fflsy 
te^b * ttfcffif fSOIilJW^S^S^te^ A 5 ffifS^S 
{k«H*ffl«-e*So 

[0017] 

(m 1 ©^ss^ffi) 0 1 1 (D^mmmic 

fiS*1ft«^fi{k»HO«fi8*«eWfc^ Ufc7D y ^0 
[0 0 1 8] EI 1 Tii, 1 >*;l/Oiiia«ffi^ 1 0 



(4) 

5 

*) fi*§ 1 0, 1 1-1-1 1 - n 

tt, zftznmMim^itmm 1 5 , ^^itss i 

6-1-1 6-nfcA#£ftTffi*Hk£ft, fflB'JO^ 
fkT-*fcLT7Jj>W77 2 3, 24-l~24-n 

^ft^fftArtSfts.. 

[0 0 19] fiH*fkfc&Bi:L*V\ SSI/Hi, Hfcft 
^{kSft/cx-^X h U-A 1 4&, ISHifcATJ^^ 
r 2 5AA^)Stl§„ ATJ/W77 2 3, 2 4- 1 ~2 10 

4 - n £-f$g«£ftfc&ffi^fbr-*43cfctf A7J^-y 

772 5fc-«f«asnfcr-^t4> ^ft^ft, 

Mk«yi»3 3, 34-l~34-n, 3 5 TMHS'JK^ 

-7*y Mksft, seic, cne/^y htf^wk&aas 

T, 0fl*.tt\ *v h7-2 3 &fl'LT£32iSftSJ:5E& 

[0020] inott ( iff*) tf^pW©/^ -y 20 

MkifflfiW3 3, 3 4- 1 -3 4-ntCtt, ^-ft^ft© 
FHHtSB 15, 16-1-16 -nt'llt, 

5 0-5 3£LTA7jf 3<k9£&oTl^3 0 

[0 0 2 1] FttHHfctt, ft^ke-yhU-K 7 
U-Aft^lbx-^S, g^7U-AU-K &7U- 

[0 0 2 2] Mfc«yia53 3, 3 4 - 1 -3 4 - 

nttt, cne^ FWfS^ffl^T^^-y hit, ^ 30 

[0023] cn&©-9--rK«fflio-ffitix A7J 

* ft 5 ffiHtftr - * %Xffi»«f -T SCfctCkOttatS 
Ci^Rjf&trfe&o 02tt, MP E G tffW^fbKfc 
tt % 7 U- ifflMlff ^©Mlf tcoi^TWWf 5 tt 

[0024] MPEG e-r *ftmtX*lt. 7 U— Afift 

#{kii» (i tf^iO , M^PJffi^bilf! (Ptf? 
fccka'M^^M^fbWf (Bt?^+) 40 

#£frtffcffi*ffk#nJf£T*££o fcfcU B + * 

So 

[0 0 2 5] EI2T*tt, I If^f-Vfe.fctfPtf*?-*© 
RgK, B M + 2 7 U-AA3 ^Wlfltitfcffl^ 

fc*^©0«-e*<&o ig^kffl-ett, 1 u^f- 

•vfc.ta'ptfi'^-v {±«#f k£ ft tcmw e> 3 7 u- a 
jwfflfijflLfc&KSjssft, ^(Dmimmtmmcoyu 

— A^^Er'J tegEttS ft, ^aiBii«l©#fiHlB«l tLXW.*§ 
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[00 2 6] MP EGtDi/T.-rA^SfbM^T'fi, tfr 
*Rtf^*^a{kUTfi2S-r*B»K:» ^t^tfitS'J 

(D T S : D e c o d i n g Time Stamp) 
fc^f^t^J (PTS: Presentatio 
n Time Stamp) ^ MPEGv-XfAM 

■pffissfts^y^wnRt^a^kbTeai^fts. 

[0 0 2 7] EI2©T-tt, I tr^ + fccfctfP tf^ + 
©PTStt, Kt?*?- + ©DT StettLT3 7U— A8H 
HO«flH^ttiPllLTJ¥tB , rSo fl&ftfcPTS 
tt, ^cffi*|fb£ftfc I tf^-vi^cttPfcf^-^D 

[0 0 2 8] LfrU H3C^TJ:5»c % I t?7^+£ 
fcti P If *f- + KR$ftfcB If + ffiKKSE 
MWT'&tK — IttfCtiftjiEOi^fCDT Sfr£PT S 

gfcttPtf^f-wcttbT, *o*Kt?F#{kSftfci t? 

^^+Sfc(iPe^^ + ^A7J$ft/c^7\ *OHIfc 

^sftSBtr^f-voa^sfte-rso lot, 01 ©®i 

HttO'^'y MbMS9$3 3-£tt, I SfcttPk?*?-* 

x-h t m$fic 2 << a 7s * > t^h-ssi- * c # tr 
-f, ^o^tcRf^k^ftfc 1 tf^f--v^rcttptf^^+ 
ommtT- * # i# & ft s * T'©^it{b3iS^4 use 

[0 0 2 9] $ 1 ©^Mm:fcnt3ti5IB£®<k=glT< 

tt, I e**+*fcttPe*f- + ©7U-Afflra*flF# 
fkSB 1 5fre>1H' KfilfHtUT/^-y Mfc«yia5 3 3 
tcAJJf Sci^JcO, &I tf^-viifcttPfc^^ 
©ft^bT-*tft#?>ft3t|S|P#K:, /*7'y MktcMS 

51^^ C5ct§<y^7h fke <£ rf ^Sf kffla^Bj* 

[00 3 0] Sfc, I £*f-+3:fcttPe*f-+0iai(H 
*fiTfC-£i:tTl?F^k%fT45«^tt, ^<D-^ffi^ 

CT^tt^k^Htj^^y-y hfk5aag(5 3 3icA^f5 

[00 3 1] M P E G 2 tfr^ftHfikfC&^T 

tt, bi 4 fc^-r* 5 few^k^ftfe 1 7 u-a^/tn-t 

^mffltt, 2 7-f-;I/K**«/^tt3 7-<— ;bKi;ffi«K 
KSfSCfctfRjflgfc&oT^So ^ftf*> 7U-Ail 
^2 4 H 7 U-AJ^Sd 3 0 H z © 

3«»-e7U-AU-hg»LT&^£H£««^teffl 

t^ft£ 0 c©ii^, i tr^f-+*fcttPt:f ^-+©ia 

PH*»t-3£fcLT^kLfe«*K:43^Tti, *I tf 
^^+tfcttPtf^^ + ©DT S t P T S ©PalPitt-^ 

ffi tt * e. f , Mis l /cfflEiafc j; 0 -»{c zmitwm* 

[0032] mi onsfiflg»j<:i3^s««^fi{k«B"r 

tt, I tf*f- + *fcttPe^ + ©7U-AraiH%ffiF^ 
fk7U-A©«^-rS^B%#iaLfe7-r-;UK»fcL 
T, Mfi^fk^H 1 hfk«LSlgI5 3 3 (C 
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7 

1MKWfBfcLTA#fSCfcK:J:?>, m^<D^Mim 
i; S C fc ft O <fr v Mkfc J: tf ££{M&Stf "TCI 

fc&So 

[0 0 3 3] H7ti, MPEG ^^^r^mt^f—^^^f 
fCB8feSi'-'5r:/*'Vy2\ I t^f-VC^SS 1 5tf 

^^■+aso*tt-e^-^tbfctr^f-+*^ (go 

P:Group Of Pictures) lc|Wfc>S G 

[0 0 3 4] 0 7K^Lfcf8j£cDf^kT-*te, ®Jli 
«ft*f{k8«l 5fr&HtfjSft* AA^77 2 3^ 
Lt;^7 Mk«iaS3 3fcArt£ttS,, /V5r-y Mkffl: 
3353 3tc&, 07K^LfcJ;9ftl»J#©T— 
^W7r 2 Sfr^A^ftSfcfcUc, iiH«ft*HkS 
Bl5fr6«, I tf^^ + SfcttPe^^ + «D7U-A 

runs, &s^«\ ^<Dy\y-L?sm^nmtyu-h(D 

-V YWLfc&W&Z'thTsZy? (DT S, PTS)£D» 20 
ttJ%tfi/\ 0 7fc^Lfc«J#©r-*5>JftV*5r«y hit* 

saaa54 3KUtfj£ftSo 

ffi 5 1S«£Sfk8H0«j££«M& WK^ L fc 7*a -y * 0 

t*&s„ 

[o o 3 5] lasfcfe^T, hi tra-awctiiBi--?? 

^HrttU SftS»#KO^T8MirrSo fftb^> El 
5 £;SLfctiM«£S{fcgHW\ I&BHt&tW^ft^k 
SI 15, 1 6-l~16-nfc/V5nyMkJ!iaaS3 30 
3, 3 4-l~3 4-n*fMffl*SJMW»6 0*£«L 
Tl^S. Sfc, 01 •e^L,fcftH«fc±tfSJ s ft*tfcS 
Hi 5, 1 6-1-1 6-n?0ffiF*f{k€>IR»c1#6ttS 
■tf-f FWBRli, S«SI56 O^Lt^^'y Mk«*aa53 
3, 3 4- 1 -3 4-n'Y*Jj , fS<fc?fcft-3TVSo 

T?<&So 

[0 0 3 6] 06iCfc^T, m 1 tm-ffiftlcttffl-tt 

5*ttu gftssMNco^TBHirf s„ t&t>%, m 40 

F1ff$K©-?*>> Wfc*-rAX*:/:/©J|£tfiK:<&gft 
15, 1 6- 1 ~ 1 6-nfrP>tH??-f Scfc-plc&oTl^ 
[0 0 3 7] HTtC^LfcMPEGtTr^W^kr-* 

[0 0 3 8] 08T'fis ->-^>X'N>y2"fc:M^T:x- 50 
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1f-T-**£fifkLT^S„ coa— *fr-*K:{i» 
MifrDl e*^+$fctiPe*?-+0lffll»teHl 1 bS^5 

^-^fcJEUT^A^^T^WWfcftftSo l kf 
* f - -v $ fc » p tr * f- ^ ©HPS # i mtL<o t/- >t y x * 

£ti, c©^££ffl^T£S{kJI&£f£MtScfctfoJ 
ST?* So 

[0 0 3 9] 09Tii, GOP'N'y^ClH^Ti— tf- 
r-$f*£fi{kLTVS. COa-tf^-^fcfcfc, luii 
co I b;**+$fcttPe**+0ISIHfcKfcS'<9y- 

+srcttPkr^f-voraiH* t GOP#(ft*i:Bri£i:*Ov 
*ffl^T^a{tiUi*ffia-rsii4:3b<pifliT*So 

[0 0 4 0] 01OT*«\ tf^-vvy^fiiigE^Ta- 
1f— 7*-**^SLTV^S. COi— tfr-*fcti, Hij 
iSOl if^f-v^fcfiPtr^f-voraHltKtoS/^^ 
-*«*S-$ftT^T, /<*-y MkJ&aaU 3T'(i, CO 

/<7*-*tcisi;T*^A;**>:/©8W*fTft$o c 

< £^AX£>7V)B[ttt#«fcftS 0 

[0041] mwbrc£oic ±iaai^6»3 

^OlWB*^{kSH 15, 1 6 - 1 - 1 6 - n Vfi 

mi* s micm h ns , xt>m t t ©ib&** 

-^SS*^ KttfRfcLT, /*ir'y Mkfflaa53 3 . 3 
4-l~34-n^X*U '<fr-y Mk*&aW3 3, 3 
4- 1-3 4-nT'li, A2)Sft;fc/<5*-*fi:fiEoT 
JB#* AT, * > fCDWmtS C ? C fc T\ &mtT- 
$ <Ds\ >y ^"gp^^rr U Tt# P> ftfc«& i: K * -f A 7, 

^y^*wtH-rs«£*o*sttt«-rsfc, fiiiffi-p© 
^fiftjoa^^scfc^nififcttSo int. nmt 

my is- Lommt^Mmcs. c s t>\ *%tw?& $ << 

i:ftS„ 

[0042] vimw 3 ? FWffiti, w 

^{br- ^ £ Jtfi[0«B^ 6 A7J L T t «fe 1/ ?5^ib 

7*-*fc : pft£a{tLTA2rrscfcfc:«k'j, ^-F9 

x 7 ^tffiBS-f kf S C fc t njflll? ife S . 
[0 0 4 3] KF^k^5^-*tt» 7b-LW4t®)W 

tcgfk-rsfeo-e&ftii, n^^-y hfksaasp^A* 

s/iv'-^yxwuic— ^©fccoT'^nii, 100 

SCfcT'SIMiT-feSo 

[0044] <MA<v$kMWM) m\ n*, 



(6) 

9 

L/c7*n-y^0T i ££o &*s, 01 Hcfcl>T, 0 1 t 

[0 0 4 5] El 1 1 tCfe^T, /*5r-y Mb«iaa53 3, 
3 4 - 1 -3 4-n, 35 Rtf^Sf k«L98P 4 3 

[0 0 4 6] fflfflSM 8H\ AAA777 2 3« 2 4- 
1-2 4 -n, 2 5teA*SttfcfiF*ffl;T--*®>W7 10 

h x h y -h<»mmm%i<o $ -r = > ^as^rtt ? t ox- 

T- Atj S ft £ g^OfiHtfbr- tKDXJi'^yyr 
ll«81t 7 r i: &o It** 5 

[ o o 4 7 ] a 1 2 a, a i i (Dmmzmtmwom® 
(@i i ©^^Btftfcgfrfr?, &cf, £*fbSH7 20 

fbigBl5, 1 6- 1 - 1 6-n WHb^ftTtfJ^S 
nfcfKfl*, SP©«F#fbT-^ 19, 2 0-1-20 
-nRl^E-OflfiOD-r-* 1 4(4, ^ft^ft£Hf{bx-2 
AJJ^BlOO, 101-1 — 101-ru 102 fC A 
7J$ft, DMAnyho-7 110, 111- 

1 — 1 11 -n, 1 1 2lcJ;0« fnftl, f-^/U 
12 0*^LT, ^^y^t'J 1 2 5±£*M$ft3A 
A;^777l 3 0, 131-l~131-n, 1 3 2^\ 
£%mZft% 0 30 
[0 0 4 8] lsjM£ftfc&'r-2«, 0 l l ©/*5r«y h 
{b&LSSI$3 3, 34-l~34-n, 3 5 tgMitmm 
35 4 3 0«&tCfflS"r3JaSttf|5*HI5C PU 1 40fe 
J:(;V7h7i7 1 4 1 fcfcD, ^ffifb^-y bXh'J- 

i»tsa*n, ^y^y 1 2 5fi©W7j;W7 7 1 

5 OtC#tjA$n5 0 

[0 0 4 9] (±17^^7 7 1 5 Otcffit&Sftfc^-tt'fb 
fcf-y hX h 'J-At4, DMA37hD-7l6 1ta 

x-^/^x 1 2 o^^LTtH^j^s i 6 2^eesi 

®s\£tiiJT£ft%<, 40 
[0 0 5 0] CCtf, C P U 1 4 043«fctf^S{bV7 h 
7i7 14 1 ii, ^Sfbt'-y bXhy-AOtUjjtr-y h 

A^ft^bx-£©ft^{b7U-A3&££7yhLT, 

? ax ? >7 , 0jg*ra*fT 4 ^ %m*m z&iiaz-o 

TV^. C P U 1 4 0fe<fctf^B{fcV7 h^i7 1 4 1 
(i, 0 1 1 cDfilffllgM 8, t&5$£>l2] 1 8 ©ffijffllgfl 3 4 9 

[0 0 5 1 ] 0 1 3*5ctU'0 1 414, /W77£ffll->fc 50 
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(54) INFORMATION MULTIPLEXER 
(5 7) Abstract: 

PROBLEM TO BE SOLVED: To provide an information multiplexer by which 
information coding, transmission, decoding real time management, and synchronization 
multiplexing of individually coded information are easily conducted. 

SOLUTION: A coding parameter obtained by coding moving image data is given to a 
packet processing section 22 separately from coded data of input moving image, and the 
packet processing section 33 processes coded data into packets. Thus, based on the 
coding parameter 50, time management information for decoding and reproducing 
coded data of a moving image is calculated and it is added to a packet of the coded data 
of the moving image, a multiplex processing section 43 applies synchronization 



multiplexing to voice coding data or the like and the result is outputted to a 
transmission line via an output buffer 45. 
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CLAIMS 
[Claim(s)] 

[Claim 1] A multiplexing means to Paquette-ize the coded data of a dynamic 
image, and audio coded data, and to multiplex them at least, An output means to 
output the multiplexing data which were multiplexed with this multiplexing means 
and obtained, An input means to input the coding parameter obtained in case 
dynamic-image data are encoded is provided. Said multiplexing means 
Information multiplexer characterized by computing the time-of-day-control 
information on decode and playback of the coded data of said dynamic image, 
and adding and carrying out multiplex [ of it ] to Paquette of the coded data of 
said dynamic image based on the coding parameter inputted with said input 
means. 

[Claim 2] Said coding parameter is information multiplexer according to claim 1 
characterized by including the parameter of rearrangement to the order of a 
display of decode from the order of the coding frame of a dynamic image. 



[Claim 3] Said coding parameter is information multiplexer according to claim 1 
characterized by including the parameter about the display period of the coding 
frame of a dynamic image. 

[Claim 4] The coding parameter inputted with said input means is information 
multiplexer according to claim 1 characterized by carrying out multiplex to the 
coded data of said dynamic image, and inputting. 

[Claim 5] Two or more inputted buffers which accumulate the coded data of a 
dynamic image, and audio coded data at least temporarily, respectively, A 
multiplexing means to Paquette-ize the coded data accumulated in each of two 
or more of these buffers, and to multiplex it, An output means to output the 
multiplexing data which were multiplexed with this multiplexing means and 
obtained is provided. Said multiplexing means Information multiplexer 
characterized by determining the timing which starts multiplexing processing 
based on the occupation of said buffer of the coded data from which transition of 
the occupation of said buffer can serve as time base among the coded data 
accumulated in said buffer temporarily. 

[Claim 6] Two or more inputted buffers which accumulate the coded data of a 
dynamic image, and audio coded data at least temporarily, respectively, A 



multiplexing means to Paquette-ize the coded data accumulated in each of two 
or more of these buffers, and to multiplex it, An output means to output the 
multiplexing data which were multiplexed with this multiplexing means and 
obtained is provided. Said multiplexing means Information multiplexer which 
carries out counting of the coding frame number, and is characterized by 
determining the timing which starts multiplexing processing based on these 
enumerated data by analyzing said coded data. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the information multiplexer which 
carries out packet multiplexing of the encoded dynamic-image bit string, the 
encoded voice bit string, and the coding bit string of further others, and is 
transmitted. 
[0002] 

[Description of the Prior Art] MPEG-1 (ISO/IEC1 1 172) which used the are 
recording media of comparatively a low bit rate as the target as an international 
standard of the compression coding method of a dynamic image, 
next-generation digital broadcasting, and VOD (Video On Demand) or -- 
MPEG-2 (ISO/IEC13818) from which the application to DVD (Digital Video Disk) 
etc. is expected etc. -- it is known. 

[0003] By the MPEG coding method, an image, voice, and other data streams 



(data stream) are encoded independently, those coded data are Paquette-ized, 
respectively, and the method which performs Paquette multiplexing is adopted. 
Moreover, between a transmitting side and a receiving side, a synchronization is 
established by transmitting each time stump of Paquette's receiving-side hitting 
time, decoding time of day, and display time of day by the frequency more than 
fixed. 

[0004] Each time stump of decoding time of day and display time of day usually 
changes with each image and coding parameters of each voice, difference with 
these time stumps always fixed in the case where divide into the coding frame of 
fixed spacing and coding is carried out to entry sequence it is possible to 
update by adding a value. However, in MPEG video coding, the order of a 
display may differ from the order of coding, and the relation may change 
dynamically. In such a case, in addition of constant value, renewal of a time 
stump becomes impossible and it is necessary to perform suitable renewal of a 
time stump from the header information of the coded data inputted etc. 
[0005] Furthermore, in a transmitting side, without causing the breakdown of 
overflow of the input buffer in a receiving side, an ANTA flow, etc., it must 
perform selection of the order of Paquette multiplexing, and multiplex [ of a time 



stump ] so that a synchronization may be established correctly. 
[0006] Usually, in the multiplexer which consists of hardware, these time stumps 
are calculated based on the time base of the multiplexing processing section. 
The time base of the multiplexing processing section can operate to them and 
asynchronous, when synchronizing with the clock of a transmission line or the 
input source. Usually, the multiplexing structure of a system changes greatly with 
a synchronization/asynchronous one of these clocks. 
[0007] 

[Problem(s) to be Solved by the Invention] By the MPEG coding method, only 
from the header information of the coded data inputted, in order to compute each 
time stump of decoding time of day and display time of day, it must have delay of 
the maximum of the gap with the order of a display, and the order of coding, and 
header information of coded data must be analyzed one by one. In the real time, 
this amount of delay cannot be disregarded in the transmission system which 
performs coding, transmission, and decode. 

[0008] The information from each information source, such as a dynamic image 
and voice, is encoded according to an individual, it asks for a coding data stream, 
and it becomes possible [ time of day, the coding number of bits, and a coding 



rate ] to connect 1 to 1 completely with the information multiplexer which 
assumed the synchronization to single time base in all processings until it 
obtains the multiplexing bit stream [-izing / carried out multiplex / of them / and / 
them / the bit stream / 1 ]. Therefore, the count of selection of the order of 
Paquette multiplexing, a time stump, etc. in multiplexer becomes possible 
[ calculating logically ] based on the imagination time base obtained from a 
coding data stream. 

[0009] In such information multiplexer, multiplexing processing becomes 
realizable only by software processing, and the system construction of it which is 
low cost and is flexible becomes possible to the exclusive hardware for real-time 
management. 

[0010] On the other hand, in the multiplexing processing by the software 
processing without real-time management, it is required that the throughput of 
multiplexing processing should be quicker than an actual code-ized rate enough 
at least. Therefore, multiplexing processing will be performed burstily and it will 
be accompanied by the big jitter between imagination time base and real time 
base on software. 

[0011] Although it is usually necessary to absorb this jitter by the transmission 



buffer, when performing coding and a decryption using real-time transmission, 
making the amount of buffer delay small as much as possible is called for. 
Moreover, in a receiving side, it is not desirable to take the large input-buffer 
delay beyond the need for jitter absorption. Therefore, input-buffer delay is 
minimized in a receiving side, and when it assumed starting the usual decode 
actuation from the input time of day of a coding data stream and the advance of 
the transmission start time of a multiplexing bit stream takes place for the gap 
with the virtual time base of the multiplexing section, and real time of day, 
possibility of causing the underflow in a transmission line or a receiving side 
arises. 

[0012] Then, this invention is made in view of the above trouble, and aims at 
offering the information multiplexer which can decrease in number the 
multiplexing delay which arises for the analysis of header information by 
inputting the coding parameter about a dynamic image, voice, the decryption 
time of day of other coded data, display time of day, etc. into the multiplexing 
section independently of the header information. 

[0013] Moreover, this invention aims at offering the information multiplexer which 
does not need complicated hardware while it can perform real-time management 



easily and prevents the above-mentioned underflow by presuming real time of 
day from the condition of the input buffer of the multiplexing section, and 
controlling multiplexing bit stream sending-out initiation. 
[0014] 

[Means for Solving the Problem] A multiplexing means for the information 
multiplexer of this invention to Paquette-ize the coded data of a dynamic image, 
and audio coded data at least, and to multiplex, An output means to output the 
multiplexing data which were multiplexed with this multiplexing means and 
obtained, An input means to input the coding parameter obtained in case 
dynamic-image data are encoded is provided. Said multiplexing means By 
computing the time-of-day-control information on decode and playback of the 
coded data of said dynamic image, and adding and carrying out multiplex [ of it ] 
to Paquette of the coded data of said dynamic image based on the coding 
parameter inputted with said input means In order to compute the 
time-of-day-control information on decode and playback of a time stump etc., the 
multiplexing delay produced in the analysis of header information can be 
decreased. 

[0015] Moreover, two or more buffers into which the information multiplexer of 



this invention was inputted and which accumulate the coded data of a dynamic 
image, and audio coded data at least temporarily, respectively, A multiplexing 
means to Paquette-ize the coded data accumulated in each of two or more of 
these buffers, and to multiplex it, An output means to output the multiplexing 
data which were multiplexed with this multiplexing means and obtained is 
provided. Said multiplexing means By determining the timing which starts 
multiplexing processing based on the occupation of said buffer of the coded data 
from which transition of the occupation of said buffer can serve as time base 
among the coded data accumulated in said buffer temporarily Complicated 
hardware is not needed, while being able to perform real-time management by 
software easily and preventing the underflow in the receive buffer by the side of 
decode. 

[0016] Moreover, two or more buffers into which the information multiplexer of 
this invention was inputted and which accumulate the coded data of a dynamic 
image, and audio coded data at least temporarily, respectively, A multiplexing 
means to Paquette-ize the coded data accumulated in each of two or more of 
these buffers, and to multiplex it, An output means to output the multiplexing 
data which were multiplexed with this multiplexing means and obtained is 



provided. Said multiplexing means By carrying out counting of the coding frame 
number, and determining the timing which starts multiplexing processing based 
on these enumerated data by analyzing said coded data Complicated hardware 
is not needed, while being able to perform real-time management by software 
easily and preventing the underflow in the receive buffer by the side of decode. 
Therefore, the information multiplexer which can perform easily the synchronous 
multiplex one of informational coding, transmission, real-time management of a 
decryption, and the information encoded according to the individual can be 
offered. 
[0017] 

[Embodiment of the Invention] One operation gestalt of this invention is 
explained with reference to a drawing. 

(1st operation gestalt) Drawing 1 is the block diagram having shown roughly the 
configuration of the information multiplexer concerning the 1st operation gestalt 
of this invention. 

[0018] Drawing 1 shows the case where the dynamic-image signal 10 of one 
channel, the sound signal of n channels, and the data signal of one channel are 
multiplexed. It is inputted into dynamic-image coding equipment 15, a voice to 



digital converter 16-1 - 16-n, respectively, and encodes, and the sound signal 
11-1 of two or more [ 10 or ] dynamic-image (video) signals - 11-n are inputted 
into an input buffer 23, and 24-1 - 24-n as coded data according to individual, 
respectively. 

[0019] Coding is not needed or the already encoded data stream 14 is similarly 
inputted into an input buffer 25. It is Paquette-ized according to an individual by 
the Paquette-ized processing section 33, 34-1 - 34-n, and 35, and further, these 
Paquette is multiplexed in the multiplexing processing section 43, and, as for the 
data stored in each coded data and the input buffer 25 which were accumulated 
in an input buffer 23, and 24-1 - 24-n temporarily temporarily, a multiplexing bit 
stream is obtained, respectively. This multiplexing bit stream 44 is transmitted 
through a network according to predetermined timing, after being accumulated in 
an output buffer 45 temporarily. 

[0020] The coding parameter obtained in case a dynamic image and voice are 
encoded by each coding equipment 15, and 16-1 - 16-n inputs into a dynamic 
image (video) and the Paquette-ized processing section 33 of a sound signal, 
and 34-1 - 34-n as side information 50-53. 

[0021] It is possible for the parameter in connection with a coding bit rate, frame 



coded data length, a display frame rate, the display period of each frame, and 
the prediction structure of video coding etc. to be included in side information. 
[0022] In the Paquette-ized processing section 33, and 34-1 - 34-n, 
Paquette-izing, calculation of a time stump, the decision of the order of 
multiplexing, etc. are made using such side information. 

[0023] Moreover, a part of such side information can also be extracted by 
carrying out grammar analysis of the coded data inputted. Drawing 2 is for 
explaining the processing sequence and the list of the frame in MPEG video 
coding. 

[0024] Coding which combined the coded image in a frame (I picture), the 
forward prediction coded image (P picture), and the both-directions 
predicting-coding image (B picture) in MPEG video coding is possible. However, 
when B picture is included, coding sequence differs from a display order. 
[0025] In drawing 2 , it is an example at the time of using the prediction structure 
where two B pictures always enter between I picture and P picture. In this case, 
in a decryption side, after I picture and P picture carry out three-frame period 
progress from the decrypted time of day, they are displayed, they are memorized 
by the frame memory of decryption equipment in the meantime, and are used for 



decryption processing as a reference image of a prediction image. 
[0026] By the system multiplex specification of MPEG, in case video and voice 
are multiplexed and transmitted, it is multiplexed and transmitted to the header 
information to which the time of day (DTS:Decoding Time Stamp) which should 
be decoded, and the time of day (PTS: Presentation Time Stamp) which should 
be displayed are specified by MPEG system specification. 
[0027] In that of drawing 2 , PTS of I picture and P picture adds and computes 
only the time amount of a three-frame period to DTS of this picture. Moreover, 
obtained PTS can be used as DTS of I picture encoded next or P picture. 
[0028] However, as shown in drawing 3 , it can change into arbitration and, 
generally the number of B pictures inserted into I picture or P picture cannot 
compute PTS from DTS as mentioned above. Usually, when I picture or P 
picture encoded by the degree is inputted to a certain I picture or P picture, the 
number of B pictures contained between them is decided. Therefore, in the 
Paquette-ized processing section 33 of the dynamic image of drawing 1 , I or P 
picture cannot calculate a time stump to an input and coincidence, and 
multiplexing delay until the coded data of I picture encoded by the degree or P 
picture is obtained will be produced. 



[0029] In the information multiplexer in the 1st operation gestalt, a time stump 
required for Paquette-izing is easily computable, and Paquette-izing and 
multiplexing processing are attained, without producing the above-mentioned 
multiplexing delay at the same time the coded data of I pictures each or P picture 
is obtained by inputting frame spacing of I picture or P picture into the 
Paquette-ized processing section 33 as side information from coding equipment 
15. 

[0030] Moreover, when encoding considering spacing of I picture or P picture as 
always fixed, it is also possible to input the constant value into the Paquette-ized 
processing section 33 for example, before coding initiation. 
[0031] Furthermore, in MPEG 2 video coding, the period which displays one 
frame encoded as shown in drawing 4 can be set as the 2 field or the 3 fields, 
and arbitration. This is used, when carrying out frame rate conversion and 
displaying the source of a movie with a frame period of 24Hz with a television 
vessel with a frame period of 30Hz. In this case, when spacing of I picture or P 
picture is encoded as always fixed, generally multiplexing delay is produced for 
the reason in which constant value did not become spacing of DTS and PTS of I 
pictures each or P picture, and mentioned it above. 



[0032] In the information multiplexer in the 1st operation gestalt, Paquette-izing 
and multiplexing processing are attained by inputting into the Paquette-ized 
processing section 33 as side information from dynamic-image coding 
equipment 15 as the number of the fields in consideration of the period when a 
coding frame displays frame spacing of I picture or P picture, without producing 
the above-mentioned multiplexing delay. 

[0033] Drawing 7 is what showed an example of the DS of MPEG video coded 
data, and consists of the sequence header in connection with the whole video 
sequence, a GOP header in connection with the picture set (GOP:Group Of 
Pictures) which carried out grouping in the unit of 15 picture extent which starts 
in I picture and a header of each picture, and coded data. 

[0034] The coded data of a configuration of having been shown in drawing 7 is 
outputted from dynamic-image coding equipment 15, and is inputted into the 
Paquette-ized processing section 33 through an input buffer 23. While the data 
of a configuration as shown in drawing 7 are inputted into the Paquette-ized 
processing section 33 from an input buffer 23, from dynamic-image coding 
equipment 15 Frame spacing of I picture or P picture, Or the value expressed 
with the number of the fields in consideration of the period when a coding frame 



displays the frame spacing is inputted as side information. Based on this side 
information, a time stump (DTS, PTS) required for Paquette-izing is computed, 
and if the data stream of a configuration of having been shown in drawing 7 is 
Paquette-ized, the computed time stump will be added and it will be outputted to 
the multiplexing processing section 43. 

(2nd operation gestalt) Drawing 5 is the block diagram having shown roughly the 
configuration of the information multiplexer concerning the 2nd operation gestalt 
of this invention. 

[0035] In drawing 5 , the same sign is given to the same part as drawing 1 , and 
a different part is explained. That is, the information multiplexer shown in 
drawing 5 possesses the control section 60 which controls a dynamic image and 
audio coding equipment 15, 16-1 - 16-n, and the Paquette-ized processing 
section 33, and 34-1 - 34-n. Moreover, the dynamic image shown by drawing 1 
and a voice to digital converter 15, and the side information acquired in the case 
of coding by 16-1 - 16-n are inputted into the Paquette-ized processing section 
33, and 34-1 - 34-n through a control section 60. 

(3rd operation gestalt) Drawing 6 is the block diagram having shown roughly the 
configuration of the information multiplexer concerning the 3rd operation gestalt 



of this invention. 

[0036] In drawing 6 , the same sign is given to the same part as drawing 1 , and 
a djff eren t part is explained. Namely, the information multiplexer shown in 
drawing 6 multiplexes data required for especially calculation of a time stump etc. 
in each coded data among the side information shown with the 1st operation 
gestalt, and outputs them from coding equipment 15, and 16-1 - 16-n. 
[0037] It is also possible to multiplex user data after each header in the DS of 
MPEG video coded data shown in drawing 7 , for example. The example of 
multiplexing of the user data in that case is explained with reference to drawing 8 
- drawing 10 . 

[0038] In drawing 8 , user data is multiplexed following a sequence header. The 
parameter in connection with spacing of the above-mentioned I picture or P 
picture is contained in this user data, and a time stump is computed in the 
Paquette-ized processing section to it according to this parameter. Spacing of I 
picture or P picture is fixed through the sequence of one unit, and when not 
using the above-mentioned frame rate conversion, it is possible to reduce 
multiplexing delay using this technique. 

[0039] In drawing 9 , user data is multiplexed following a GOP header. The 



parameter in connection with spacing of the above-mentioned I picture or P 
picture is contained in this user data, and a time stump is computed in the 
Paquette-ized processing section to it according to this parameter. When 
spacing of I picture or P picture serves as adjustable per GOP and it does not 
use the above-mentioned frame rate conversion, it is possible to reduce 
multiplexing delay using this technique. 

[0040] In drawing 10 , it is carrying out multiplex [ of the user data ] following the 
picture header. The parameter in connection with spacing of the 
above-mentioned I picture or P picture is contained in this user data, and a time 
stump is computed in the Paquette-ized processing section 43 to it according to 
this parameter. In this case, it becomes always computable [ without multiplexing 
delay / a time stump ] about the prediction structure of arbitration. 
[0041] As mentioned above, as explained, according to the 3rd operation gestalt, 
multiplexing of time stumps, such as an MPEG system, sets from the above 1st 
to a required multiplex system. The parameter which shows the relation of the 
entry sequence and the order of coding which are obtained in case information, 
such as a dynamic image and voice, is encoded by coding equipment 15, and 
16-1 - 16-n, the parameter showing the display duration of a frame, etc. are 



made into side information. It inputs into the Paquette-ized processing section 
33, and 34-1 - 34-n. In the Paquette-ized processing section 33, and 34-1 - 34-n 
It becomes possible to realize multiplexing processing by low delay as compared 
with the conventional method which computes a time stump based on the 
information which analyzed the header unit of coded data and was acquired by 
what I will set calculation of a time stump for one by one according to the 
inputted parameter. Usually, although delay of several frames arises 
theoretically when computing a time stump by analyzing coded data, in this 
invention, the theoretic delay for calculation of a time stump is not produced. 
[0042] Although the side information containing a coding parameter may be 
inputted from a path independent of coded data, it can also simplify hardware by 
multiplexing to coded data beforehand and inputting into it. 
[0043] A coding parameter can change dynamically per frame, and that can be 
inputted into the Paquette-ized processing section one by one, and it can realize 
in it being fixed per sequence and inputting that into the Paquette-ized 
processing section only once to one sequence. 

[0044] (4th operation gestalt) Drawing 11 is the block diagram having shown 
roughly the configuration of the information multiplexer concerning the 4th 



operation gestalt of this invention. In addition, in drawing 11 , the same sign is 
given to the same part as drawing 1 , and a different part is explained. 
[0045] In drawing 11 , multiplexing of a time stump and control of the order of 
multiplexing are performed one by one according to imagination time base in the 
Paquette-ized processing section 33, 34-1 - 34-n, 35, and the multiplexing 
processing section 43. 

[0046] A control section 48 supervises an input buffer 23, 24-1 - 24-n, and the 
buffer occupation of the coded data inputted into 25, and performs timing control 
of transmitting initiation of the multiplexing bit stream which notifies the time of 
multiplexing processing of coded data, and initiation of sending out to the 
multiplexing processing section 43. For example, the timing from which the 
occupation of each input buffer turned into a buffer occupation equivalent to the 
time of day of the data transfer initiation timing logically drawn from the 
input-buffer occupation of the coded data of the voice inputted at a fixed rate is 
notified to the multiplexing processing section 43. 

[0047] Drawing 12 shows more concretely the configuration of the important 
section (it is the part surrounded by the dotted line of drawing 11 , and is 
hereafter called the multiplexing section 47) of the information multiplexer of 



drawing 11 . The data 14 of coding equipment 15, the dynamic image encoded 
and outputted by 16-1 - 16-n, the audio coded data 19, 20-1 - 20-n, and others It 
is inputted into the coded data input unit 100, 101-1 - 101-n, and 102, 
respectively. Further by DMA controller 110, 111-1 - 111-n, and 112 
Respectively, it is transmitted to the input buffer 130 constituted on main 
memory 125, 131-1 - 131-n, and 132 through a data bus 120. 
[0048] By CPU 140 and software 141 which manage the processing actuation 
equivalent to the function of the Paquette-ized processing section 33 of drawing 
1 1 , 34-1 - 34-n, and the 35 and the multiplexing processing section 43, each 
transmitted data is changed into a multiplexing bit stream, and is written in the 
output buffer 150 in main memory 125. 

[0049] The multiplexing bit stream written in the output buffer 150 is outputted to 
a transmission line by DMA controller 161 from an output unit 162 through a data 
bus 120. 

[0050] Here, by counting the output number of bits of a multiplexing bit stream, in 
the 5th operation gestalt which updates virtual time base and is mentioned later, 
CPU 140 and the multiplexing software 141 count the coding frame number of 
each input coded data, and manage the processing which performs renewal of a 



time stump etc. CPU 140 and the multiplexing software 141 constitute the control 
section 48 of drawing 11 , and the control section 349 of below-mentioned 
drawing 18 . 

[0051] Drawing 13 and drawing 14 are drawings for explaining the synchronous 
approach which used the buffer. In drawing 13 , in 170 and 174, a transmitting 
(output) buffer and 172 show a transmission line, and, as for coded data and 171, 
173 shows the receiving (input) buffer. This transmission buffer and receive 
buffer are also called a smoothing buffer. 

[0052] Coded data 170 is inputted into a transmission buffer 171, and after it is 
accumulated temporarily and delayed, it is inputted into a receive buffer 173 
through a transmission line 172. The transmitted coded data is outputted after 
delay is similarly added in a receive buffer 173. 

[0053] Here, by controlling so that the sum of the amount of delay in a 
transmission buffer 171 and the amount of delay in a receive buffer 173 
becomes always fixed, the total amount of delay after coded data 170 inputs into 
a transmission buffer 171 until it is outputted from a receive buffer 173 becomes 
always fixed, and the data transfer which synchronized by transmission and 
reception becomes possible. 



[0054] timing-chart ** of transmission and reception when the smoothing buffer 
of drawing 13 is used for drawing 14 -- it carries out. In drawing 14 , the image 
data of one frame of a dynamic image is encoded at intervals of predetermined, 
the coded data from which the amount of signs differs for every frame is 
generated, and it is sent to a transmission buffer 171 one by one. 
[0055] 190 shows the residence time in a transmission buffer 171, and 192 
shows the residence time in a receive buffer 173. Although the residence time in 
a transmission buffer 171 and the residence time in a receive buffer 173 may be 
changed in time from the principle of a smoothing buffer, respectively, the total 
amount of delay during the transmission and reception which are the sum is 
******(ed) so that it may become always fixed. 

[0056] By the coding method by which the amount of signs of a frame unit is 
changed like MPEG video coding, smoothing of the amount of modulation codes 
which used the transceiver buffer 171 is performed. As mentioned above, in a 
transmitting side, after coded data is inputted, it is outputted after delay by the 
transmission buffer 171. Since each size of a transmission buffer 171 and a 
receive buffer 173 is the specified finite value, the amount of delay in each buffer 
must be controlled strictly not to cause an underflow and overflow with each 



buffer. 

[0057] Drawing 15 shows the input-buffer occupation in the multiplexing 
processing section 43 of each coded data of the transmitting side at the time of 
starting. Each coded data of an image and voice has coding delay peculiar to a 
coding method and a coding parameter, is inputted into an input buffer 130, and 
131-1 - 131-n, and after passing through the time amount equivalent to the 
further above-mentioned transmission buffer delay, it multiplexes and it is sent 
out. 

[0058] In drawing 15 , 201 is the coded data of an image, 202 and 203 are audio 
coded data, respectively, and the example in the case of multiplexing these 
three data is shown. By drawing 15 , all coded data show transition from the 
initial state of the input-buffer occupation in the case of being inputted into the 
multiplexing section 47 at a fixed rate. Here, since the coded data of an image 
has delay of a transmitting side, it is necessary to start transmission to the timing 
of time of day 204, and the audio coded data 202 and 203 shall start multiplexing 
and transmission from the timing of time of day 206 and 207, respectively. 
[0059] since input spacing of the frame which be a candidate for coding , the 
amount of signs which encode it and be obtain , a coding rate , etc. be 



completely connect with 1 to 1 like the coded data of a dynamic image when 
change the amount of a frame unit of signs , when software realize the function 
of the multiplexing section 47 , management of the real time ( real time of day ) 
become usually difficult only by perform smoothing by the above-mentioned 
buffer . 

[0060] That is, if sending out is started by advance by the transmitting side (** 
which does not perform timing control of transmitting initiation of the multiplexing 
bit stream of this invention), possibility of causing the underflow in a receiving 
side for the gap with the virtual time base of the multiplexing section 47 and real 
time of day will arise. 

[0061] Thus, in the case of the coded data of a dynamic image, management of 
the real time can also be said to be the supervisory control of the transmit timing 
of the coded data in the transmitting side which holds actual screen spacing with 
the decoder of a receiving side, and can decode coded data by taking into 
consideration the amount of signs of one frame (screen) which is 1 coding unit. 
[0062] Then, the input-buffer occupation of a certain specific dynamic image or 
the audio coded data 202, for example, the coded data of voice which is always 
inputted with a fixed bit rate, is supervised, and timing control of transmitting 



initiation of the multiplexing bit stream which starts multiplexing processing and 
sending out is performed from the time of becoming a buffer occupation (for 
example, drawing 15 input-buffer occupation 205) equivalent to the time of day 
(for example, drawing 15 time of day 204) of the data transfer initiation timing 
logically drawn from this fixed bit rate. Thereby, the underflow in a receive buffer 
which was mentioned above becomes possible [ protecting ]. 
[0063] Drawing 16 shows transition from the initial state of an input-buffer 
occupation in case the coded data 210 of a dynamic image is burstily inputted 
into the multiplexing section 47, and drawing 17 shows transition from the initial 
state of an input-buffer occupation while an input rate carries out time variation 
to the multiplexing section 47, in case the coded data 220 of a dynamic image is 
inputted by it. 

[0064] Although it is difficult to presume real time of day from a buffer occupation 
when an input is not a fixed rate When there is voice coded data (212 of drawing 
16 , 222 of drawing 17 ) of accompanying inputted at a fixed rate, it is made to be 
the same as that of explanation of drawing 15 . From the input-buffer occupation 
of the multiplexing section 47 about the coded data The buffer occupation 
equivalent to the time of day (for example, drawing 16 time of day 214 and 



drawing 17 time of day 224) of the data transfer initiation timing drawn logically 
(- for example, timing control of transmitting initiation of the multiplexing bit 
stream which starts multiplexing processing and sending out is performed from 
the time of becoming the input-buffer occupation 215 in drawing 16 , and 
becoming input-buffer occupation 225) in drawing 17 . 

[0065] (5th operation gestalt) With the configuration of the multiplexing section 
47 shown in drawing 12 , since the input buffer 130 of each coded data, 131-1 - 
131-n, and 132 are constituted on main memory 125, the data will become 
transparent from software. Therefore, it is good also as a configuration which 
draws the current time for counting the coding frame number and controlling 
multiplexing initiation timing instead of measuring the occupation of the coded 
data in an input buffer. 

[0066] Drawing 18 is the block diagram having shown roughly the configuration 
of the information multiplexer concerning the 5th operation gestalt. Drawing 18 
shows the case where two or more programs which are the groups of a dynamic 
image and voice are encoded and multiplexed to coincidence. Here, the 
programs Ch1 and Ch2 which consist of the dynamic images and voice which 
are encoded in the real time, and beforehand encode, and it carries out multiplex 



[ of the program of a total of three Ch(s) of the program Ch3 currently recorded 
on are recording media ] to coincidence. The detailed configuration of the 
important section (it is the part surrounded by the dotted line of drawing 18 , and 
is hereafter called the multiplexing section 347) of the information multiplexer of 
drawing 18 is the same as that of drawing 12 . 

[0067] It is inputted into dynamic-image coding equipment 315, a voice to digital 
converter 316, dynamic-image coding equipment 317, and a voice to digital 
converter 318, respectively, and encodes, and the dynamic-image (video) signal 
300, a sound signal 301 and the dynamic-image signal 302, and the voice image 
303 are inputted into input buffers 323, 324, 325, and 326 as coded data 
according to individual, respectively. 

[0068] The program data 304 read from are recording media are similarly 
inputted into an input buffer 327. Moreover, coding is not needed or you may 
make it input the already encoded data stream 305 into an input buffer 351 
similarly. 

[0069] each coded data accumulated in each input buffer temporarily 
respectively — Paquette-ized processing section 333- it is Paquette-ized 
according to an individual by 337 and 353, and further, these Paquette is 



multiplexed in the multiplexing processing section 343, and a multiplexing bit 
stream is obtained. This multiplexing bit stream 344 is transmitted through a 
network according to predetermined timing, after being accumulated in an output 
buffer 345 temporarily. 

[0070] Paquette-ized processing section 333- 337 and 353 -- respectively -- 
being alike the data analysis sections 333a, 334a, 335a, 336a, 337a, and 
353a which extract header units, such as coded data inputted from each input 
buffer, and are analyzed are provided. 

[0071] Based on the analysis result in these data analysis section, a control 
section 349 counts the coding frame number of a dynamic image, and 
multiplexing initiation timing control, i.e., real time of day, is computed, and it 
notifies the timing which multiplexes and transmits to the multiplexing processing 
section 343. In addition, about each program of Ch1-Ch3, when the time base is 
independent, the virtual clock in multiplexing software is calculated 
independently, respectively, and timing control of multiplexing initiation is 
performed for every program. Moreover, in a common program, time base shall 
also communalize the virtual clock in multiplexing software, and shall perform 
timing control of multiplexing initiation. 



[0072] As explained, in case it carries out synchronous multiplex [ of coded data 
encoded according to the individual, such as a dynamic image and voice, and 
the other data ] according to the operation gestalt of the above 4th, as mentioned 
above, a control section 48 About the specific image which supervises the 
occupation of the data of each input buffer and is inputted with a fixed bit rate, or 
audio coded data The time of day of the data transfer initiation timing logically 
drawn from the fixed bit rate For example, the time of becoming the buffer 
occupation (for example, drawing 15 input-buffer occupation 205) which is 
equivalent to time-of-day 204) in drawing 15 is notified to the multiplexing 
processing section 43. (-- the multiplexing processing section 43 ** which can 
realize multiplexing processing which can be equal also to real-time 
transmission by starting multiplexing processing and sending out according to 
the timing notified from the control section 48 only by software By realizing the 
multiplexing section 47 only by software, it becomes unnecessary to perform 
timing control by complicated hardware, and a system configuration can be 
simplified. Moreover, customize by software becomes easy and correspondence 
of it is flexibly attained to various applications. 

[0073] In case it carries out synchronous multiplex [ of coded data encoded 



according to the individual, such as a dynamic image and voice, and the other 
data ] according to the 5th operation gestalt of this invention, moreover, a control 
section 349 Based on the analysis result of the data analysis section of each 
packet-ized processing section, real time of day is computed by counting the 
coding frame number of a dynamic image, and it is notified to the multiplexing 
processing section 343. The multiplexing processing section 343 ** which can 
realize multiplexing processing which can be equal also to real-time 
transmission by starting multiplexing processing and sending out according to 
the timing notified from the control section 349 only by software By realizing the 
multiplexing section 347 only by software, it becomes unnecessary to perform 
timing control by complicated hardware, and a system configuration can be 
simplified. Moreover, customize by software becomes easy and correspondence 
of it is flexibly attained to various applications. Furthermore, it is effective even if 
it uses from the above 1st, combining the 5th operation gestalt suitably. 
[0074] 

[Effect of the Invention] As explained above, according to this invention, the 
information multiplexer which can decrease in number the multiplexing delay 
produced in order to compute a time stump by analyzing header information can 



be offered by inputting the coding parameter about a dynamic image, voice, the 
decryption time of day of other coded data, and display time of day into the 
multiplexing section independently of the header information. 
[0075] Moreover, while according to this invention being able to perform 
real-time management easily and protecting the underflow of the receive buffer 
by the side of a transmission line or decode from the specific image inputted with 
a fixed bit rate, the input-buffer occupation of audio coded data, or the coding 
frame number in an input buffer by controlling multiplexing bit stream 
sending-out initiation for them using the changed value to real time of day, the 
information multiplexer which does not need complicated hardware can be 
offered. Thus, according to the information multiplexer of this invention, it can 
perform easily the synchronous multiplex one of informational coding, 
transmission, real-time management of a decryption, and the information 
encoded according to the individual. 



DESCRIPTION OF DRAWINGS 
[Brief Description of the Drawings] 

[Drawing 1] The block diagram having shown the configuration of the information 
multiplexer concerning the 1st operation gestalt of this invention. 
[Drawing 2] Drawing for explaining the processing sequence and the list of the 
frame in MPEG video coding shows the case where the number of sheets of B 
picture is immobilization. 

[Drawing 3] Drawing for explaining the processing sequence and the list of the 



concerned frame in MPEG video coding shows the case where the number of 
sheets of B picture carries out adjustable. 

[Drawing 4] Drawing for explaining the display period of each coding frame in 
MPRG video coding. 

[Drawing 5] The block diagram having shown the configuration of the information 

multiplexer concerning the 2nd operation gestalt of this invention. 

[Drawing 6] The block diagram having shown the configuration of the information 

multiplexer concerning the 3rd operation gestalt of this invention. 

[Drawing 7] Drawing having shown an example of the DS of the MPEG video 

coded data in the information multiplexer concerning the 1st operation gestalt 

and the 2nd operation gestalt. 

[Drawing 8] Drawing having shown an example of the DS of the MPEG video 
coded data in the information multiplexer concerning the 3rd operation gestalt. 
[Drawing 9] Drawing having shown other examples of the DS of the MPEG video 
coded data in the information multiplexer concerning the 3rd operation gestalt. 
[Drawing 10] Drawing having shown the example of further others of the DS of 
the MPEG video coded data in the information multiplexer concerning the 3rd 
operation gestalt. 



[Drawing 11] The block diagram having shown the configuration of the 
information multiplexer concerning the 4th operation gestalt of this invention. 
[Drawing 12] The block diagram having shown the configuration of the important 
section of information multiplexer in the detail more. 

[Drawing 13] Drawing for explaining the synchronous approach using a 
transceiver buffer (smoothing buffer). 

[Drawing 14] Drawing for explaining actuation of a smoothing buffer. 
[Drawing 15] In drawing for explaining the timing control of the multiplexing 
processing concerning this invention, and transmitting initiation, all coded data 
show transition from the initial state of the input-buffer occupation in the case of 
being inputted into the multiplexing section at a fixed rate. 

[Drawing 16] Drawing for explaining the timing control of the multiplexing 
processing concerning this invention and transmitting initiation shows transition 
from the initial state of an input-buffer occupation in case the coded data of a 
dynamic image is burstily inputted into the multiplexing section. 
[Drawing 17] Transition from the initial state of an input-buffer occupation while 
an input rate carries out time variation to the multiplexing section, in case the 
coded data of a dynamic image is inputted in drawing for explaining the timing 



control of the multiplexing processing concerning this invention and transmitting 
initiation is shown. 

[Drawing 18] The block diagram having shown the configuration of the 
information multiplexer concerning the 5th operation gestalt of this invention. 
[Description of Notations] 

15 [ The Paquette-ized processing section, 43 / The multiplexing processing 
section, 44 / » A multiplexing bit stream, 45 / -- An output buffer, 47 / 48 The 
multiplexing section, 60 / -- A control section, 347 / -- Multiplexing section. ] - 
Dynamic-image coding equipment, 16-1 - 16-n A voice to digital converter, 23, 
24-1 - 24-n, 25 -- An input buffer, 33, 34-1 - 34-n, 35 



